The microscopic origin of thermodynamic entropy in isolated systems 
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A microscopic understanding of the thermodynamic entropy in quantum systems has been a mystery ever 
since the invention of quantum mechanics. In classical physics, this entropy is believed to be the logarithm 
of the volume of phase space accessible to an isolated system Qj]. There is no quantum mechanical analog to 
this. Instead, Von Neumann's hypothesis for the entropy 0] is most widely used. However this gives zero for 
systems with a known wave function, that is a pure state. This is because it measures the lack of information 
about the system rather than the flow of heat as obtained from thermodynamic experiments. Many arguments 
attempt to sidestep these issues by considering the system of interest coupled to a large external one, unlike the 
classical case where Boltzmann's approach for isolated systems is far more satisfactory. With new experimental 
techniques, probing the quantum nature of thermalization is now possible |3, 4]. Here, using recent advances 
in our understanding of quantum thermalization |5- 10] we show how to obtain the entropy as is measured from 
thermodynamic experiments, solely from the self-entanglement of the wavefunction, and find strong numerical 
evidence that the two are in agreement for non-integrable systems. It is striking that this entropy, which is closely 
related to the concept of heat, and generally thought of as microscopic chaotic motion, can be determined for 
systems in energy eigenstates which are stationary in time and therefore not chaotic, but instead have a very 
complex spatial dependence. 



The emergence of thermodynamics from underlying micro- 
scopic motion is still unclear. One of the hardest concepts 
to understand is that of entropy. Classically, the microscopic 
state of a system can be thought of as a single point in phase 
space and as it evolves, this point wanders in this space, fill- 
ing up more of it as time progresses. Boltzmann's hypothesis 
relates the entropy to the volume that can be filled, and shows 
how microscopic motion determines large scale thermody- 
namic properties For a generic, that is a non-integrable, 
system, ergodicity implies that this volume will be the hy- 
persurface of constant energy, but for integrable systems, the 
volume accessible depends on initial conditions so that such 
systems will not thermalize. 

There are many different approaches and definitions for un- 
derstanding entropy 111 itl . but here we will be concerned with 
the experimental thermodynamic definition that is obtained, 
for example, by doing calorimetry measurements as related 
to the flow of heat. Classically, these thermodynamic mea- 
surements are not influenced by knowledge of the microscopic 
state of a system. For example, computer simulations are of- 
ten used to predict thermodynamic quantities, such as entropy. 
However microscopic knowledge of the system allows one to 
extract work out of it by the well known example of Maxwell's 
demon. This would violate the second law of thermodynam- 
ics unless we correspondingly reduce the entropy to take into 
account this microscopic information Therefore classi- 
cally there are two distinct uses of the word entropy: (/) ob- 
tained by performing macroscopic thermodynamic measure- 
ments, which we will call the thermodynamic entropy, and 
(//) a measure of the lack of available information about a sys- 
tem's state. 

The quantum mechanical situation is far less clear. Knowl- 
edge of the system through observation does alter the state 
of the system, so that it is not apparent if the thermodynamic 



entropy is altered by this process. For a system with density 
matrix p , the Von Neumann entropy S'va? = — Tr(p In p ) which 
is identically zero for any system in a pure state, including an 
energy eigenstate. Because the state of the system is known 
completely, this reduces its entropy to zero as in case (//). But 
what does this do to entropy (/)? If we do thermodynamic 
measurements on systems in pure states, that is, where the 
state is known completely to the experimenter, does this alter 
what is measured? 

Furthermore, the quantum ensemble definition of entropy, 
case (/), is simply related to the density of states of the energy. 
But if the system is in a single energy eigenstate, it might ap- 
pear to be impossible to get the density of states. This would 
argue that thermodynamic entropy measured for systems in 
pure states, would be different than for a system coupled to 
an external environment. Such questions until recently were 
purely theoretical but now with the emergence of experiments 
that study thermalization of isolated quantum systems or lack 
thereof p','3], a proper understanding of the microscopic ori- 
gins of entropy has become increasingly important. 

Here we find convincing evidence that for pure states, there 
is a way of defining an entropy as in case (/) for a non- 
integrable quantum mechanical system with a large number of 
degrees of freedoms which is identical to the thermodynamic 
definition and give an explicit prescription for its construction 
from a knowledge of the eigenstate alone, thus successfully 
linking the thermodynamic entropy with its underlying micro- 
scopic origins. Throughout this work we will be concerned 
with the thermodynamic limit of a large number of degrees of 
freedom, A^, and will not consider non-extensive corrections 
to the entropy for finite systems. Furthermore in this limit, 
the microcanonical and canonical ensembles give identical re- 
sults iil]. Therefore we can freely choose which ensemble to 
consider. 
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Statistical mechanics is concerned mainly with computing 
the time average of observables that depend on only a few 
variables, and relates this time average to an average over an 
ensemble. Thus for a system with average energy {E), statis- 
tical mechanics posits the relation between the time average 
of some observable O and its average over the microcanonical 
ensemble of states, the latter being far easier to compute: 



(1) 



where the bar on the left hand side denotes a time average and 
the microcanonical density matrix at energy Eq is 
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and A/i is much greater than the average distance between 
neighboring energy levels but much less than the macroscopic 
energy scale £0, and .yVstates is the number of terms in the sum. 
In quantum mechanics, the fluctuation in the energy can be 
large [5] but in practice is taken to be small so that {E) is a 
good measure of the energy. The way to see the connection 
between the underlying quantum evolution needed on the left 
hand side of Eq. [T]of a generic system and the ensemble meth- 
ods of statistical mechanics has recently become much better 
understood. The idea is the "Eigenstate Thermalization Hy- 
pothesis" (ETH) isl-lztl: for large A^, the expectation value of 
an observable in an energy eigenstate becomes equal to the 
microcanonical average at the same energy, that is 



Tr(p£()) ^ Tx{p,nicro,Ed) 



(3) 



where p is the density matrix for the wavefunction \E) ?it en- 
ergy £, Pe = \E){E\. Put more simply, the expectation value 
of O will vary very little between neighboring energy levels 
for large A^, implying that the expectation value of O in any 
energy eigenstate is the microcanonical average (at that en- 
ergy). There is strong numerical evidence and analytic argu- 
ments to support this for a variety of systems f3SlB\. Not ah 



systems obey this hypothesis Ill2lll3[l . In particular, integrable 
systems, do not do so. Note that Pe ^ pmicm.E, as there a large 
number of constants of motion, even for generic Hamiltoni- 
ans, such as projection operators lfTill that violate Eq. [3] This 
is why the choice of observables satisfying this equation is 
restricted. 

Below we shall extend these ideas to understanding en- 
tropy. In the thermodynamic limit statistical mechanics 
posits the formula for the thermodynamic entropy Sthermo = 
-Tr(p„„cro In Pmiero) where p,„icro could be equivalently re- 
placed with the thermal density matrix in the canonical en- 
semble in the limit that we are considering. We want to see if 
this can be calculated from the properties of the wavefunction 
itself without any recourse to ensembles. 

We consider homogeneous systems for simplicity with 
short range interactions and start by following the standard 
textbook scenario: the system is divided into two parts, the 
larger part, B, can act as a bath in contact with the smaller 



one. A, that is our system of interest. If the complete system 
starts out in a pure state, then by doing a partial trace over B, 
the reduced density matrix for A, pA = Tr^p becomes mixed 
because it is entangled with B. This is part of a common but 
non-rigorous argument for how a canonical distribution of en- 
ergies emerge for the smaller system A, where the temperature 
T is given by the statistical mechanical relation between T and 
the average energy. If the complete system was actually inte- 
grable this argument fails and thermalization does not occur. 
The entanglement entropy is defined as 



5£,„(A,B) = -Tr(pAlnpA). 



(4) 



which has been studied in detail for a wide variety of sys- 
tems and is very important in the study of quantum informa- 
tion theory lITsll . black holes |16] and quantum phase transi- 
tions |17^. This will also be important in what follows. 

If instead we considered the complete system to already 
be described by the microcanonical density matrix, then 
PA.micro = '^^BPmicro must bchavc with Canonical statistics 
even for integrable systems, and the entanglement entropy in 
thatcaseis5£„f,„„o-o(A,B) = - i:x{pA.micriMPA.micro) which is 
the statistical mechanical expression for the entropy. This en- 
tropy is what one measures experimentally in thermodynamic 
experiments. Thus 



^thermo 



{A) — SEnl. micro {A,B) = -Tr{pA .micro In Pa .micro ) (5) 



This argument is very similar to most treatments of entropy in 
textbooks OlSll however there is more evidence than that this 
to support it. Rigorous analysis [19] gives an canonical en- 
tanglement entropy for one dimensional models in agreement 
with the thermodynamic results |20]. 

This does not connect thermodynamic entropy with the 
wavefunction of a system because the right hand side of Eq. |5] 
involves the microcanonical average and is similar to the right 
hand side of the ETH as written in Eq. |3] but differs because 
of the nonlinear logarithmic factor which is not an observable 
operator Thus we ask whether we can extend the ETH to 
quantities of this form; that is, for a generic quantum system 
in an energy eigenstate can we replace Pa. micro with p^? In 
other words is 



,(A) = -Tr(pAlnpA)-i 



(6) 



This form was recently hypothesized using other less direct 
analytical arguments 12 lil . Although this is the thermody- 
namic entropy for a subsystem of A + B, the entropy is ex- 
tensive so we can obtain the entropy for the full system by 
adding the entropies for individual subsystems together. In 
the particular case of a homogeneous system, the above equa- 
tion can easily give us the thermodynamic entropy per unit 
volume. Thus entropy defined this way is a measure of self- 
entanglement. 

Now we employ exact diagonalization on a number of sys- 
tems to determine if Eq. |6]is satisfied, and how it scales with 
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system size. Because system sizes are still far from the ther- 
modynamic limit we will minimize finite size effects by ask- 
ing equivalently using Eq. |5] does 



-Tr(pAlnpA) = -Tl{pA,,nwnAnpA,,„icmV- 



(7) 
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FIG. 1. (a) The entanglement entropy 5£:„/(A,S) (Eq.|4ll, for a non- 
integrable system of 6 hard core bosons on 27 sites where A has 4 
sites and B has 23. The blue curve « = 1 is obtained from the entan- 
glement entropy of individual energy eigenvectors. The dashed green 
curve is the microcanonical entropy obtained by averaging the den- 
sity matrix over « = 100 neighboring eigenvectors. The dashed red 
curve is obtained from the entanglement entropy of a wave function 
that is a random superposition of 100 neighboring levels. The light 
blue dashed curve is the curve n = I smoothed over 100 neighboring 
energy levels, (b) The same as in (a) but for an integrable system 
t' = V' = 0. 



We study systems that have been previously shown to give 
rise to energy eigenstate thermalization. Following some of 
these previous studies 1, 8; - 10.1 we consider spinless fermions 
and hard core bosons on a one dimensional lattice with near- 
est neighbor (NN) and next nearest neighbor (NNN) hopping, 
and interaction. The hopping strengths for NN and NNN are t 



and t', likewise we denote the interaction strengths V and V'. 
We study these systems for parameters where they are known 
to obey ETH and for integrable parameters where they do not. 
Following previous work, we use periodic boundary condi- 
tions in the subspace with wavevector k—1, rather than k — Q 
to avoid a parity symmetry. Throughout this work we set the 
energy scale to have t = h~ 1. 

For both the fermions and hard core boson cases, t' — V' — 
0.96 which is non-integrable and should obey the ETH. We 
calculate the entanglement entropy for each energy eigenstate 
for a range of different bath and system sizes. Our largest 
size was N — 27 lattice sites with A^^ = 6 particles. We trace 
over the bath sites and calculate the entanglement entropy Eq. 
m and do this as a function of the number of lattice sites of 
A, denoted m. For these cases, we plotted the entanglement 
entropy as a function of m. As is well known |22] Se„t{m) — 
Sent{N — m) so for m > N/2 the entanglement entropy must 
go back down to zero. As we display in the supplementary 
information in the case of hard core bosons, it is very close 
to linear behavior over a substantial range of m, for the non- 
integrable case, as is expected due to the extensive nature of 
entropy. 

However the behavior for integrable systems, t' — V' ~ 0, 
is more erratic and the linearity depends much more strongly 
on the eigenvector Similar results were also obtained for the 
fermion model as well as is shown in the supplementary ma- 
terial. 

We directly checked to see if Eq. |7] is supported numer- 
ically. We calculated the left hand side of this equation for 
different values of m and because of the linearity of the en- 
tanglement entropy with m, the results are insensitive to this 
value. In Fig. [T] it is shown for m — 4. We also calculate 
the microcanonical reduced density matrix by taking the par- 
tial trace of Eq. |2l We average the reduced density matrix 
over 100 neighboring eigenvectors, and use that to calculate 
the entanglement entropy as done on the right hand side of 
Eq. |7] The result for 6 hard code bosons on 27 sites is shown 
in Fig. \Tla). The blue line is no microcanonical averaging 
(« — 1) and the green dashed curve is the microcanonical av- 
erage over n = 100 neighbors. The two curves are very close 
to each other in most of the range where the density of states 
is high and one has eigenstate thermalization. The light blue 
dashed curve takes the entanglement entropy with no micro- 
canonical averaging and averages it over 100 neighboring en- 
ergy levels. The dashed red line will be discussed below. 

In contrast. Fig. \llh), for the integrable case t' — V' — 0, 
one sees that the fluctuations are much larger and that the mi- 
crocanonical average is much further from the entanglement 
entropy for individual eigenstates. Similar results are seen in 
the one dimensional spinless fermion system. To quantify this 
difference further, we examine the standard deviation of fluc- 
tuation of the entanglement entropy for both the integrable and 
non-integrable cases. 

We calculated the standard deviation g{S) of the entangle- 
ment entropy Sent{A,B) around the maximum of the curves in 
Fig. [T]for different lattice sizes, 16, 18, 20, 25, and 27, all 
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FIG. 2. The fluctuations in tiie entanglement entropy for botii non- 
integrable (generic) and integrable systems of 6 hard core bosons (B) 
but with a different number of lattice sites, and the same quantity 
for the spinless fermion model (F). The standard deviation of the en- 
tanglement entropy Sent {A, B) is plotted against the density of states. 
This is done around the point of maximum entanglement entropy. 



with 6 particles in Fig. |2] g{S) is computed over 100 neigh- 
boring energy levels, and this is shown as a function of the 
density of states on a log-log plot for both the hard core boson 
models (B) and spinless fermions (F), and for both the non- 
integrable and integrable cases. As is apparent, the integrable 
fluctuations are substantially higher. This is also the case for 
observables as has been shown previously f?]. The behav- 
ior as a function of the density of states and system size is 
qualitatively different. The fluctuations are diminishing much 
more rapidly in the non-integrable case. This is similar to 
slower diminution of fluctuations of observables predicted for 
integrable versus non-integrable systems, where in the former 
case we expect a power law decrease as a function of the num- 
ber of degrees of freedom and in the latter case it is predicted 
to be an exponential [5]. 

The above studies provide excellent numerical evidence 
for the equivalence of the thermodynamic entropy of non- 
integrable systems with the entanglement entropy when the 
wavefunction is in an energy eigenstate. We also studied two 
dimensional hard core bosons and fermions and reached the 
same conclusion (see supplementary materials). Now we ask 
what happens with more general initial states. If one starts the 
system off in a state with fluctuations in energy in a window, 
then we can ask how this evolves for long times. We expect 
thermalization of observables, and that the entropy should also 
be the thermodynamic entropy. We can test to see if this is the 
case numerically. 

Over long times, the energy eigenvector components of 
non-integrable wave functions will have random phases [23J. 
Therefore we test this case to see if the entanglement en- 
tropy still looks like the microcanonical entanglement entropy. 
In Fig. [TJa) the dashed red line computes the entanglement 
entropy at each energy by using wavefunctions that are the 
superposition of 100 neighboring energy eigenvectors with 
Gaussian amplitudes and random phases. As can be seen, this 



matches the predictions of the microcanonical entanglement 
entropy very well, and the same conclusion is also reached in 
the fermion case. The closeness of the reduced density matrix 
to the canonical result is expected for general initial conditions 
where many energy eigenstates are present [ 24)1 . Our above 
analysis shows that for generic systems, when the system is 
put in an energy eigenstate, we still obtain the microcanon- 
ical entropy, which is crucial, as otherwise thermodynamics 
would fail in this important case. 

Therefore the equivalence of thermodynamic and the en- 
tanglement entropy for long times should be correct, starting 
from a wide range of initial states. 

We can now answer the questions that we originally posed. 
Although the statistical mechanical entropy is most simply 
understood from the density of states, for a generic Hamilto- 
nian, the self-entanglement of a single energy eigenstate can 
be used to obtain the same result as well. Knowledge of neigh- 
boring levels is not necessary as this entropy as we have seen, 
is a property of a single energy energy eigenvector Going 
back to our earlier comparison with classical physics, we can 
now answer our original question. A complete knowledge of 
the system's quantum state, does not affect its behavior with 
respect to macroscopic measurements of the entropy. An ex- 
perimenter who measured a systems's state precisely, will in 
subsequent measurements obtain the same answers as some- 
one who is not made privy to this information, despite the 
latter describing the system in a mixed state. For long times, a 
system in a pure state and one in an statistical ensemble have 
identical thermodynamic entropies in the limit of large sys- 
tems. This is because the entropy in such experiments mea- 
sures the system's self-entanglement, not the lack of knowl- 
edge of it. 

We thank M. Rigol and M.R. Peterson for useful discus- 
sions. Haibin Li is supported by State Scholarship Fund(No. 
2010833088) 
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We start by considering two kinds of one dimensional lattice systems, hard core bosons (HCB) and 
spinless fermions (SF). We choose periodic boundary conditions and add both nearest-neighbor (NN) and 
next-nearest-neighbor (NNN) hopping and interactions. Following the notation of Santos and Rigol Ml, the 
Hamiltonian for bosons Hb and for fermions Hp are 

L 



HB = f^ [-t {bjbi+i+h.c}j +Vn'^nli - t' (b]bi+2+h.c}j +V'n\n\^ 
1=1 



(1) 



and 



Hf = Y. [-t {fifi+i + ^-c-) + - ^' {fIfi+2 + h.c.) + V'n{ . (2) 

i=l 

Here, L is the length of the chain, bi and b] are bosonic and /, and fj are the fermionic annihilation 
and creation operators for site /. = bjbj is the boson and n{ = fjfi is the fermion local density operator. 
Hardcore bosons cannot occupy the same site, and the operators commute on different sites. The NN and 
NNN hopping strengths are respectively t and t'. The interaction strengths are V and V' respectively. We 
take n = t = V = \. 

Because of translational invariance and particle number conservation, the Hamiltonian divides into in- 

R 

dependent blocks each with fixed total momentum k. As previously pointed out [20, to avoid degeneracy 
associated with parity symmetry at ^ = 0, we take k = I and checked that the energy levels obey Wigner- 
Dyson statistics. 

Using exact diagonalization, we computed the eigenvalues and eigenvectors of the Hamiltonian in the 
k = I sector We studied the cases V' = t' = .96, and V' = t' = 0. The HCB results for the entanglement 
entropy 5e„,(m) are shown in Fig. [Ha) for the non-integrable case V' = t' = .96 and the in integrable case 
Fig. dtb) V' = t' = 0. Here the number of lattice sites N = 27, and the number of particles Np = 6. The 
five plots sample different eigenvectors and are evenly spaced as a function of the eigenvector index. The 
plots start at m = 3 because for m smaller than a correlation length, we do not expect to obtain the correct 
entropy. The corrections to linearity are well fitted with a small exponential term. For the integrable case 
(b), the behavior is much more erratic and less straight than in the non-integrable case(a). Fig. |2]we display 
the analagous results for spinless fermions with the same parameters. 

We show the same quantities as shown in Fig. 2 of the letter for spinless fermions in Fig. [3l with the 
same parameters as used for the hard core bosons. 

Because of the strong linearity in the entanglement entropy as a function of m, we can take the difference 

= Sent{^) — Sent{^) to be an estimate for the entropy of the system per lattice site. The results are very 
similar to those obtained using AS = Sent (5) — Sent (4) instead. The results for hard core bosons are displayed 
in Fig. m This difference displays the same features as the full entanglement entropy and still shows very 
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good agreement with the microcanonical results in the case of a non-integrable system, but does not agree 
well for the integrable case. 

We next consider the same quantities for a two dimensional square lattice of hard core bosons with 
periodic boundary condition. We chose the total momentum sector to be zero. Hard core bosons in two 
dimensions have also been shown to satisfy energy eigenstate thermalization [1]. We consider 6 hard core 
bosons on a 5 by 5 lattice. We choose repulsive nearest neighbor interactions V = 0.1 and a nearest neighbor 
hopping strength t = I. As above we chose A5 = Sent (4) — Sent (3) 

Fig. [5] shows the results from exact diagonalization of this model. As we found in one dimensional 
non-integrable systems, the microcanonical entanglement entropy is in very good agreement with the en- 
tanglement entropy. 



[1] M. Rigol, V. Dunjko, and M. Olshanii, Nature 452, 854 (2008). 
[2] L. F. Santos and M. Rigol, Phys. Rev. E 81, 036206 (2010). 
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FIG. 1 . The entanglement entropy for single energy eigenstates of hard core bosons shown for a number of different 
levels in the non-integrable case. The levels were chosen to be evenly spaces and are 548 (lowest), 1096, 1644, 2192, 
2740 (highest) out of a total of 10966 states in the k ~ 1 sector (a) The non-integrable case, for t' = V' = 0.96. The 
fit was to a straight line plus and exponential correction and the standard deviation of the error for each curve is less 
than 3.4 x 10"^ for all of these levels, (b) The integrable case t' = V' = 0. 
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FIG. 2. The entanglement entropy for single energy eigenstates of spinless fermions shown for a number of different 
levels in the non-integrable case. The levels were chosen to be evenly spaces and are 548 (lowest), 1096, 1644, 2192, 
2740 (highest) out of a total of 10966 states in the k ~ 1 sector (a) The non-integrable case, for t' = V' = 0.96. The 
fit was to a straight line plus and exponential correction and the standard deviation of the error for each curve is less 
than 3.1 X 10^^ for all of these levels, (b) The integrable case t' = V' = 0. 



6 



(a) 




(b) 




n=l 

n=100 micro 
n=100 super 
ave S 



FIG. 3. (a) The entanglement entropy SEnt{A,B), for a non-integrable system of 6 spinless fermions on 27 sites 
where A has 4 sites and B has 23. The blue curve « = 1 is obtained from the entanglement entropy of individual 
energy eigenvectors. The dashed green curve is the microcanonical entropy obtained by averaging the density matrix 
over n = 100 neighboring eigenvectors. The dashed red curve is obtained from the entanglement entropy of a wave 
function that is a random superposition of 100 neighboring levels. The light blue dashed curve is the curve « = 1 
smoothed over 100 neighboring energy levels, (b) The same as in (a) but for an integrable system t' = V' = 0. 
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FIG. 4. (a) The entanglement entropy difference for a non-integrable system of 6 bosons on 27 sites for the same 
system as Fig. 1 of the accompanying letter. Here AS' = Sent{^) — Sent{'i)- The blue curve « = 1 is obtained from 
the entanglement entropy of individual energy eigenvectors. The dashed green curve is the microcanonical entropy 
obtained by averaging the density matrix over n = 100 neighboring eigenvectors. The dashed red curve is obtained 
from the entanglement entropy of a wave function that is a random superposition of 100 neighboring levels. The light 
blue dashed curve is the curve n — \ smoothed over 100 neighboring energy levels, (b) The same as in (a) but for an 
integrable system f ' = V' = 0. 
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FIG. 5. The entanglement entropy difference for a non-integrable system of 6 hard core bosons on a two dimensional 
5x5 lattice. Here AS — Sent {^)— Sent {i)- The blue curve n = 1 is obtained from the entanglement entropy of individual 
energy eigenvectors. The dashed green curve is the microcanonical entropy obtained by averaging the density matrix 
over n = 100 neighboring eigenvectors. The dashed red curve is obtained from the entanglement entropy of a wave 
function that is a random superposition of 100 neighboring levels. The hght blue dashed curve is the curve n = \ 
smoothed over 100 neighboring energy levels. 



